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Abstract 

We investigate properties of a channel coding scheme leading to the minimum-possible frame error ratio when 
transmitting over a memoryless channel with rate R > C. The results are compared to the well-known properties 
of a channel coding scheme leading to minimum bit error ratio. It is concluded that these two optimization requests 
are contradicting. A valuable application of the derived results is presented. 

I. Introduction 

We consider coded data transmission over a memoryless channel with given capacity C for the case 
that the rate of the channel exceeds the channel capacity. This is a typical situation for a component 
code in a concatenated coding scheme [1]. The properties of a channel coding scheme with minimum 
average bit error ratio (BER) have been discussed in several papers (e.g. [2] and references therein). In 



^ this paper, we focus on schemes with minimum average frame error ratio (FER). The work presented in 
OO 

O this paper is threefold. As a first contribution, we discuss a coding scheme optimal w.r.t. FER and use 

.!-H rate-distortion theory to derive the properties of the end-to-end channel. Second, we present a possible 

X 

^ application of these findings. This is a lower bound on the rate when both the channel capacity and the 
tolerated average frame error ratio are specified. This leads to the most important and third contribution, 
the insight that minimum BER and minimum FER are contradicting targets which cannot be obtained by a 
single channel coding scheme. We show the consequences for the BER when the channel coding scheme 
is optimized w.r.t. to the FER and vice versa. The paper is organized as follows. Section |Il] provides 
necessary definitions and describes the transmission system. In Section UlI] we repeat the converse to 
the channel coding theorem which identifies a lower bound for reliable transmission. Section |IV] briefly 
repeats the properties of channel coding scheme with minimum BER, which was derived in [2]. In Section 
rvl ideal channel coding w.r.t. FER is introduced. An application as well as performance results for the 
non-optimized error ratio are shown in Section |VIl 



II. Transmission Setup and Definitions 



An information source delivers source symbols u[i], £ = l,...,k, from a binary alphabet. These 
symbols are realizations of the random variables U[£] which are assumed to be independent of each other 
and have identical distributions. In other words, H{U[£]) = 1 holds, where H(-) denotes the entropy of 
a random variable. In the following, we denote the j-th element of the set of possible source words by 
u^^\ j = 1, . . . , 2^^ and u[i], £ = 1, . . . ,k denotes the i-th entry in a vector u of length k. An encoder 
of rate R = k/n is used to transform binary source vectors u of length k into vectors x. These vectors 
contain channel symbols and are of length n. The vectors x are transmitted over the channel and received 
as vectors y of length n, cf. Figure \\\ Note that we do not assume any special properties of the channel 
except for being discrete and memoryless (discrete memoryless channel, DMC) and meeting the capacity 
C. A corresponding channel decoder uses the received vector y to generate soft-output estimates of u, 
denoted by v. Binary quantization of v yields ii. As stated above, we are interested in channel coding 
schemes that minimize the BER and PER, respectively, when measured over the end-to-end channel. 
Here, the end-to-end channel corresponds to the channel transmitting to t; if soft-decision output is 
required and ii otherwise, cf. Figure [H 
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Fig. 1. Transmission scenario for signaling over a DMC of capacity C 



We define the average bit error ratio for a given position £ as BER^ = Pr(?7[£] ^ U[£]), £ = 1, . . . ,k 

k 

and the average bit error ratio in a codeword as BER = -^Yl BER^ = E^jBER^}. Additionally, a 

e=i 

tolerated average bit error ratio BERt is introduced. This ratio is technically equal to BER, as it is also 
measured between u and ii, but BERt is a user-defined threshold variable. Henceforth, we tacitly assume 
BERt < 0.5. Similarly, we define the average frame error ratio as the probability that the received frame 
differs from the transmitted one, i.e. PER = ViiU ^ U) where equality of two vectors is given if all 
elements in the two vectors are equal. Alike BERt, we consider the tolerated average frame error ratio 
PERt- 

I{U] V) denotes the total mutual information between vectors (frames) u and v while I{U\£]; V\£]) is 
the mutual information between an individual pair of input and output symbols. Additionally, we define 
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the average mutual information transmitted in a frame of k symbols, I{U; V) ^ I HW]; 

III. Converse to the channel coding theorem 

We repeat the converse to the channel coding theorem as stated in [3, Ch. 4]. This theorem marks the 
starting point for both our considerations on the lowest BER and PER. In [3, Ch. 4] the transmission of a 
sequence of source digits is discussed. Note that these digits can represent both single information symbols 
as well as complete source words. We represent this distinction by different alphabets and as a consequence 
we denote the length of a sequence of source symbols by L. As each digit can be taken from an arbitrary 
alphabet, we denote the sequence of source digits by a sequence of vectors, ttf = [ui, . . . ,Ul]. The 
sequence of channel digits and received digits, both of length N, are denoted by Xi — [xi, . . . ,Xn] and 
Hi = • • • 5 Vn]^ respectively. Please note that the cardinalities of the sets of channel input and output 
variables do not have to be specified, the mutual information I{X^; Y^) is sufficient. In the following, 
we consider the general error event that the source digit and the estimated digit do not coincide and denote 
the probability by Pg. Identifying the source digits from alphabets of size M — 2 and M — 2^ allows us 
to deduct information on the average error ratios of interest, i.e. the BER and the PER, respectively. 

We start by Equation (4.3.20) from [3] (Fano's inequality), which is a lower bound on the error 
probability Pe and reads 

eM(Pe) > \h{U{) - ; ). (1) 

Here, eui') is the M-ary entropy function, eM{p) = G2{p) +plog2(M — 1), and e2(-) is the usual binary 
entropy function 62 (p) = — plog2(p) — (1 — p)log2(l — p)- Further, M denotes the size of the symbol 
alphabet. 

Let us first assume that uf is a vector of binary symbols of length L, i.e. L — k and A^^ = n. In this 
case, Pe coincides with the BER and \H{U\) = 1 holds. Together with /(X"; 1^") < nC due to the 
memoryless channel and the capacity maximum of mutual information, the well known lower bound on 
BER results: 

e2(BER)>l-^C = l-^. (2) 

As mentioned above, we regard the source sequence u for a lower bound on PER as one symbol out of a 
2*^-ary alphabet, and the variables x and y represent an entire codeword and received word, respectively. 
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In block coding, each source sequence is encoded into one codeword and subsequent codewords are 
mutually independent. Therefore, L = N holds and Equation ^ reads 

e2.(FER) > ^H{U[) - ^/(Xf ; Y^) = H{U) - /(X; Y). 

With H{U) = k and I{X; Y) < nC, we obtain the corresponding result to Equation ^ for FER, 

(PER) > k-nC = k{l-^). (3) 

It is worth mentioning that Equation (O is a special case of Equation ([3]) for A; = 1. In the following, 
we will show by means of rate-distortion theory that the lower bounds in Equation ^ and Equation ^ 
can be met with equality by optimized channel coding schemes. We will denote the average frame error 
ratio measured in a system optimized w.r.t. to minimum BER by PER'. Likewise, the average bit error 
ratio measured in a system optimized w.r.t. to minimum PER will be denoted by BER'. 

IV. Obtaining the lowest possible BER for a memoryless channel with given capacity 

We investigate the transmission of data at a rate which exceeds the capacity, i.e. we consider the region 
where error-free transmission is not possible. Rate-distortion theory [4] postulates, that if an end-to-end 
average bit error ratio BERx is tolerated, a code with rate R and appropriate decoding rule exists and 
achieves an average bit error ratio BER < BERt as long as 

C 

- 1 - 62 (BERt) 

and if n ^ oo. 

We define a coding scheme (i.e. code, encoder, and decoder) with rate R = ^^^^(bert) ideal 
in terms of the average bit error ratio, iff the average bit error ratio meets the tolerated one, BER = 
BERt = (l — It was shown in [2] that Equation (U) is met with equality when signaling over a 
memoryless binary symmetric channel (BSC). 

This leads to the conclusion that the use of a coding scheme ideal w.r.t. BER results in an end-to-end 
channel which is a memoryless BSC [2]. For completeness we add that in [2] it is also shown that for an 
ideal coding scheme w.r.t. BER, I{U\ U) = I{U; V) = \liU] V) holds, i.e. soft-output has no benefit 
over hard-output and interleaving has no influence on the memoryless sequence of errors. 
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Fig. 2. Minimum FER for different block lengths and given C/R 

V. Obtaining the lowest possible FER for a memoryless channel with given capacity 

In this section we present the first contribution of this paper. We discuss a channel coding system 
leading to the end-to-end channel with the lowest possible average frame error ratio. The adaptation of 
the converse to the channel coding theorem for the FER being the error ratio is given by Equation ([3]). 

Figure [2] shows the corresponding lower bound on the FER over k for different values of C/R. The 
lower bound on C/R, specified by Equation ([3]), becomes particularly interesting when k approaches large 
values. Then, C/R > limfc^oo (l - ^^^^ - ^^^^^|^FEr) = 1 - FER, or equivalently, 

FER > 1 - C/R. (5) 

Again, we show that this bound can be met with equality by means of rate-distortion theory and by this 
it is proven that a coding scheme meeting the FER has indeed to exist [5]. To this end, let us first show that 
there exists a channel meeting inequality ([5]) with equality, namely the M-ary symmetric channel (M-SC) 
with M = 2^ . We assume that u and u denote the input and output symbols of that channel, respectively. 
The transition probabilities of this end-to-end channel are denoted as Pr(C/ = u^^ \ U = u^^^) = 1-FER 
Wj e {!,..., 2''} and Pr(C/ = | U = u^^'>) = FER/(2'= - 1), Vj, j ^ i. 

The mutual information per channel use is calculated by 

I{U;U) = H{U)-H{U I U) 



— C/R = 0.1 (o) to CIR = 0.9 «) 




Prfr/~l = [2-*...2-*l 



(6) 
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A; + ^ ^ Pr(LA = n^^^ | L/ = u^^) Pt{U = u^'^) log. 
i=i j=i 

what can be simplified to 



E Pr(?7 = I [/ = n(^')) Pr(C/ = n(*'); 



I{U;U) =k-e2k{FER). (7) 

Equation dV]) denotes the mutual information for the transmission of a whole vector. Normalized to one 
binary symbol it reads ^-^^ = 1 + iog2(l - FER) + ^ loga (^|^) , and thus for k ^ oo: 

ii^ = l-FER. 

k 

Considering the data processing theorem in the form ^^^^^ < ^ and Equation ^ in the form C/R > 
1 — FER allows us to conduct ^^^ff^^ ~ ^ ~ 1 — FER for A; — > oo. Again, by making use of rate-distortion 
theory and considering the fact that a distinct test chanel exists [5], we conclude that the lower bound 
provided in Equation ([5]) can be met with equality. 

In the following, we will review the properties of this channel. In the case of an error, the 2'^-ary 
symmetric channel maps the input to all incorrect outputs with equal probability. We therefore conclude 
that if a frame error occurs, the average bit error ratio within these frames is 0.5. Hence, the resulting 
end-to-end channel corresponds to a fully bursty channel. More strictly speaking, a block-erasure channel 
with average erasure probability FER and infinite frame length meets the bound C/R > 1 — FER with 
equality. This finding allows us to establish a coherence between the capacity and the rate, when an 
average frame error ratio FERt is tolerated. This coherence reads R < . 

We define a coding scheme (i.e. code, encoder, and decoder) with rate R = to be ideal in terms 

of the average frame error ratio, iff the average frame error ratio meets the tolerated frame error ratio 
FERt with equaUty, FER = FERt. 

We denote the obtained average bit error ratio of such a channel by BER'. There exists a straightforward 
coherence between BER' and the optimal average frame error ratio FER which reads BER' = | (l ~ ^) • 
The capacity of the fully bursty binary (end-to-end) channel, where all errors are part of very long error 
bursts, can be written as C = 1 — 2BER'. This is due to the fact that all errors are concentrated in bursts 
and within these bursts the average bit error ratio is 0.5. Reliable communication is accomplished by the 
simple rule of erasing the error bursts at the receiver side. For error detection, e.g. by means of a cyclic 



redundancy check (CRC), additional redundancy is necessary but this cost vanishes for k ^ oo. Alike 
stated in Section JVl we observe that soft information has no benefit over hard output if an optimal coding 
scheme w.r.t. minimum FER is used. Examples for such fully bursty channels can simply be generated 
by renewal burst channel models, like the model of Fritchman with a single error state [6]. For given 
BER, the capacity of such a channel is maximized when the average burst length tends to infinity and in 
this limit, the capacity equals 1 — 2BER. This entity exactly corresponds to the situation of bit errors at 
the output of a coding scheme which is ideal w.r.t. minimum average frame error ratio. Consider now an 
end-to-end channel with minimum BER. For k oo, the obtained average frame error ratio, denoted by 
FER' is given by FER' = iff BER = and FER' = 1 otherwise. When considering blocks of infinite 
length, every block is erroneous if bit errors are possible in general. 

With the results derived so far, it is straightforward to see that a channel coding scheme working in 
the region R > C cannot obtain the minimum-possible BER and the minimum possible FER with one 
channel coding scheme, cf. Figure |3(a)[ 

VI. Possible Application 

A possible application of the presented results is introduced in this section. We assume binary antipodal 
signaling (BPSK) over the AWGN channel with a channel code of given rate R. A lower bound on the 
obtainable BER is given in Equation Q, which can be rewritten to BER > (l — ^). 

This entity allows to generate the curves depicting the optimum BER obtainable by codes of given rate 
and length approaching infinity. These curves are well-known from numerous publications within the area 
of channel coding and visualize the fundamental limits for transmission at a given rate. In Figure |3(b)| 
these curves are shown for the rates R= 1/4, R= 1/2, i? = 3/4, respectively, and are labeled by BER. 
Here, the capacity of the channel is specified by the signal-to-noise ratio expressed by 10 log^Q(i?b/iVo) 
as usual. In this context, Ey, denotes the energy per transmitted bit of information and A'^o represents the 
one-sided spectral noise-power density. 

The findings presented in Section |V] allow to extend these fundamental curves to scenarios where the 
FER is used as the performance measure. Here, Equation © states the lower bound on the FER which 
can be reached by code lengths approaching infinity. Figure |3(b)| also depicts lower bounds on the frame 



error ratios for codes of rate R = 1/4, R = 1/2, and R = 3/4. These curves are labeled by FER. 
Figure |3(b)| also shows the average error ratios BER' and FER'. These curves illustrate the average bit 
error ratio for the case that the channel coding system of the end-to-end channel has been optimized for 
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(a) Visualization of the significant difference between BER and (b) BER, BER', PER and FER' for coding schemes of given 

BER' for coding schemes optimal w.r.t BER and FER, respec- rate. 

tively 

Fig. 3. Performance comparisons on BER, BER', FER and FER' 



the FER and the average frame error ratio for the case that the optimization was done for the BER, 
respectively. This confirms that channel coding schemes being ideal w.r.t. minimum BER and minimum 
FER have to be designed in quite different ways. Especially an optimization w.r.t. the BER in situations, 
when a low average frame error ratio is required as well, leads to significant performance losses. In all 
cases, the average mutual information of the end-to-end channel is given by min (C/i?, 1). In order to 
obtain minimum BER, the end-to-end channel is a memoryless BSC, whereas for minimum FER, an 
end-to-end channel with memory (to be precise, a block-erasure channel) results. 



VII. Conclusions 

We considered transmission at rates exceeding the capacity of the underlying channel. Fundamental 
insight in a threefold manner is given. First, knowledge on a coding scheme leading to an end-to-end 
channel with minimum average frame error ratio is provided. It turns out that this channel is a block- 
erasure channel, transmitting frames either correctly or in such a way that no information is transmitted 
at all. The average bit error ratio within a frame corresponding to a burst error equals 0.5. Second, it 
was shown that minimum BER and minimum FER are disparate requests to a channel coding scheme. 
The third contribution is an application. It is usual in literature to compare the BER behavior of channel 
coding schemes to information theoretic bounds; this is now also possible with respect to the FER. 
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